1. Field of the Invention
A system for mixing or compositing real-time, computer generated 3D objects and a video feed from a film camera, such as a video camera, to generate a real-time augmented reality video for TV broadcast, cinema or video games.
2. Description of the Prior Art
Over the past 20 years, there has been considerable research and commercial activity in this field; reference may be made to video insertion or augmented reality systems from companies, such as Sportvision, Inc, which has developed television viewing enhancements for sporting events, such as American football, adding a virtual first down line that viewers see super-imposed onto the pitch. Other companies that have developed systems in this area include PVI, Inc. One common feature of known systems is that they rely principally on analysing the features in the video feed from the camera to determine which part of the real-world scene the camera is pointing at; the system will have earlier built up a 3D map of that scene so that, once it knows where in the scene it is pointing at, it is able to add or composite computer generated objects (such as the virtual first down line) onto the video feed in such a way that the object's position and orientation makes it appear a natural part of the scene. One disadvantage of replying purely on the optical flow in this way is that such systems can be unreliable.
Other systems rely on pure marker-based approaches (Lightcraft Technologies for example). They require an operator to put real physical markers (1 m×1 m large) on set to be detected by the system. It is highly inefficient as it needs hours or days to set up a stage, which is something very unlikely to work for movie production. It also has many constraints as the physical markers must always remain in the field of view of their system. Scientific references may also be made to the papers cited in Appendix 1.